DOCUMENT RESOHF 



2D--109 176 



TM OOa 623 



AOTHDR 
TITJ.E . 

I 

INSTITUTION 
S.PONS AGENCY 

PO.B DftTE • ' 
NOTE 



SDES PRICE 
DESCRIPTORS 



Wilson; Barry J. • ' , 

Evaluation and "Prediction of Rated Job Performance of 

Nursing Graduates. _ . ■ ^ 

Delta Col'l.., Un^plsity Center ^ M'ich. 

Health Eesources^.dministnation (DHEW/PHS)., Bethesda, 

Md. Div. of Nursing. ,< 

[Apr 75] ; ^ 

26p.;-' Paper presented *at th4 Annual Meeting of tHe 
American Educational Research , Association 
(Washington, D. C, March 30- April 3/ 1975) 

MF-$0.76 HC-$1.95 PLUS POS€!AGE 

Associate Degrees; Evaluation ; *E valuation Criteria; 
Gra'de Point Average; *Hurses; Predictive Ability 
(Testing) ; *Predictor Variables; *Rating Scales; 
♦Task Performance ' * ' 



ABSTRACT 

de'gree nursi 
close superv 
constructed 
training ins 
Graduates al 
number* of cu 
Results also 
scholastic s 
performance, 
implementati 



The job perfo 
ng progranv. was ev 
ision of the grad 
to represent both 
titution as well 
so rated -ttheir pf 
rticuljim iarplicat 

indica^te^ no sign 
uccess (GPA and .o 

^The study provid 
on* of a basic eva 



rmance of 153 gr 
aluated by docto 
uates. The evalu 

the cur*rioului? 
as performance t 
eparatian on a s 
ions of the resu 
ificajfi^t relation 
t-ate 'Board score 
es-an example pf 
iuation^ model* 



aduatres of, an associate' 
rs and nurses working in 
at ion instrument was 
objectives of the 
riteri'a of supervisors* 
iijilar instrument/ A 
Its are discussed • 
ship bet^en measures' of 
s) and ra^ed job 

succe*s-ful / J 

Author) • . ' : 



****** ********* 




Documents acq 
materials not a,va, 
to obtain the best 
reproducibility are '"'.of ten-'encotinter^d'.'^^^^^ af.^e<its ^ Vhe quality 

of the microfiche ^;id hardcopy repf oduct ions \ES^C* fcakes '^v^ 
via the, ERIC Document' Reproduction' Service JEJ5^5\- ijEDHS is, not *" 
responsible f qr. tlife quality^.<}f - the original" ^iocum^t* Rej^rod'uctions 
supplied by EDSS/are the be.sl^ "that can made ffo'C' the original. 



* 



******************* ********ak************************:lc ******* *********** 



t-H 

O 

T—i 

Q 



Evaluation and Prediction of Rated Job Performance, of Nursing Graduates 

- .ft - , ' 

American Educational Research' Association 
1975 Annual Meeting 



us OEPARtMENtOF HEALtH, 
EOUCAtlONiWELf ARE 
NATIONAL INStltUTEOF 
~ EOUCAtlON 



ctaTEO 00 NOT NeceSSARILY R«;*''<«^ 
SENT OFMOAL NATIONAL '^^^^ ' ^ ^ 
' EOUCATtON POSITIONER POUCV^. . 



r/ 



. Presented by 
•^'OK-"'"Barry J. Wilson> Universrty of Northern Iowa 
Mr. Gene^^Packwdbd, Delta College 

r- \ • ■ 




^[^^^.^^ \e^J^u^ ^^^^^ 



{^'''^■■Z i^L^u^i^ /9 i^itW^y /^^i*-*^^'^^'^^^ 



p.. 



The Job' Performance of Nursing Gradu^ites: A Program Evaluation 

Barry J, Wilson ' 
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; ^ • ^ " APSTRACT 

The Study was. designed to assess potential strengths and weaknesses of 
nursing preparation and training as\ef1ected in job performance of nursing 
schpol graduates and also to investigate the predictive relationship of measures 
of scholastic success such as GPA and Nursing Board Scores with graduate job 
performance. ,A 62 item rating scale was constructed and designed to tap the 
following dimensions of nursing |)erformance: planning for nursing care, imple- 
menting hursing care, interpersonal relations,h1ps and communication, leadership 
aod group procedures, evaluating^ ^ind reporting nursing care, and professional* 

involvement. Input for the rating scale reflected curriculum objectives as 

^ . \ ^ 

well as a field survey of performance criteria, graduates were rated by a 

nurse and a doctor who worked in closest supervision of their job. Graduates 

completed a similar rating scale in which they were asked to nate the adequacy 

of, their preparation for the various performance dimensions. Ratings We 

obtained from a sample of 153 grajluates of the associate degree nursing program 

at Delta College, Univer^sity^ Center, Michigan. Results indicated a need for 

* 

additional clinical experience requiring total involvement of nursing students,^ 
advanced courses in^ pharmacology, anatomy, physiology, and nutrition," and 
leadership 'experience. Results also suggested no relationship between Various , 
indices of GPA and Board Scores With raided performance.' It is probable that 
rated performance in influenced by a number of personality variables. Also 
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doctors appear to perceive the perforjnance of nurses from different perspectives 
than do supervising nurses • 




WITRODUCTION 



The current stutfy was undertaken to investigate the post-graduate job 
status and performance of associate degree nursing graduates at Delta College, 
University Center, Michigan. The study was (Resigned to provide data relevant 
to the following questions: (1) What are some possible strengths and weak- 
nesses of the Delta Nursing Program as reflected in the job performance of 
nursing graduates? (2) What are the predictive relationships among measures 
of scholastic achievement such as grade point average (GPA) and State Board 
of Nursing Examinations and the* job performance of nursing graduates? Some 
assumption^' implicit in these questions include: (a) That valid and reliable 



indides of 

i 

weaknesses 



job performance can be secured; (b) That rated sti^engths and 
of job performance may reflect streng4:hs a?>d weaknesses of the 



nurs,1ng curHc(ilum; (c) That grades and Board scores are sufficiently relia- 

blp and valid to allow prediction to job performance; and (d) That factors 

.contrjibutind to good grades and high Board scores are also significant factors 
in later performance in the nursing profession. . , 
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■ • THE PROBLEM 

The task of gathering valid information regarding nursing performance is». 
not as easy one as Magnusson (1967) notes. Of central importance is the 
problem of constructing a concise definition of nursing siiccesS. It is ah 
evident fact that the nursing profession is not a. single occupation but a 
whole range of occupations and different areas of nursing place different 
"demands upon individual capabilities. What constitutes a good nurse in one 
field does not necessarily hold in another. TITe situation is further compli- 
cated in that evaluations of job^erformance depend upon the observation of 
more than one judge. Judges who rate the same individuals can have seen ' 
these individuals- in different situations and different individuals can have, 
been rated by different judges-. These considejrations indicate the problem 
of establishing an absolute criterion of nuHing' success. 

' feiven the task, of assessing the job performance of nurses, the author 
con(iucted a search for a standardized instrument to evaluate nursing performance. 
Inspecti^'on of the standard reference by. Buros (1971) revealed no such instrument 
in the area of nursing evaluation whic^ was_ designed for on-the-job* performance. 

Construction of such an instrument was therefore indicated, , ^ 

♦ * . ■» ' 

Job performance may be evaluated in e^entially three ways: by means of a 
rating scale, a checklist, or product evaluation. The latter was eliminated 
as unfeasible in this case; the second eliminated because it does not lend 
itself to quantitative analyses', tonstruction of a rating scale was therefore 
indicated:- , ■ 



* A primary consideration in the construction of the rating scale was content 
validity'.. In this'^case, given the basic objectives of the stady, content validity 
implies that the, content or items of the ins-trument accurately sample both the 
,curricular objectiv:es of -the nursing program at Delta and on-the-dob performance 
criteria. Therefore, construction of the rating scale included not only ^ cur- 
ricular objectives provided by the Division of Nursing at Delta College but also 
performance criteria collected from the^ field. 

. A good rating scale demands qualified observers who perform the rating. In 
^ the present study it was decided that one primary , source of qu^^ified observers 
should be the nurse who exercises closest supervision over theVaduate and who 
is presumeable in the best position to .evaluate ali^spects of the job performed 
by the graduate. This, person is designated' "nurse sup^isor" in this study 
although this appelatiori should not be interpretea as^ the job tttle of the' rater 
and should not be confused with the individual sd designated in a typical ^ 
hospital setting, the "nursf supervisor" in the present case is that person 
designated by the Delta graduates as "the .nurse in closest supervision of their 
work." It wa^ decided that an additional ra'ting be obtained from a doctor who 
worked closely with the graduate'. A third rater in the study is jtlhe Delta 
graduate. In this case, however, rather than rating their* own performance, 
graduates were asked to rate how they^ perceived the ad€quacy of their prepara- 
"'tion at Delta in relation td the performance criteria sampled by the rating 

scale. . - - 

In summary, the basic problems posed by the study were.i^ (iTDesign of ^ 
rating^scal.e which accurately sampVed the domain of "nursing success;" and 
■ (2) Soliciting. ratings from the best possible. sources; 



•REVIEW OF RELATED LITERATURE * 

* 

» 

•Measures of scholastic success such as GPA and, standardized test scores 
have tfaditionally played an important role in the selection of individuals 
for jobs and advanced study. This role has come under attack in recent years. 
Kirschenbaum, Simon, and Napier (1971) summarize a- number of studies investi- 
getting grades as predictors of academic and occupational performance. Amo^ig 
these, a study by Prince,. Taylor, Richards, and Jacobsen (1969) found no 
relation of grades in medical school to twenty-four performance characteristics 
of physicians although there was a slight relationship observed between grades 

■r 

X 

and doctors who contributed to the professional literature. -Kappell (1962) 
reported a, slight positive, correlation between college grades and final salaries 
attained\y employees of the American Tel 6phbne and Telegraph Company but no 
relat-ionship with other performance criteria. Barr (1961) summ?ir1^[ed 'thirty- . 
three stqdies in the area of prediction of teacher effectiveness, and found a 
median correlation of .09 (negligible) between supervisor ratings and college 
GPA of teachers. On the other hand, there are studies which do report higher . 
I relationships between grades and later performance (e.g. Breckenridge, 1932). 
When grades are used as predictors, data^can be obtained as to the accuracy 



,;.vwith which they do predict. This is something which ihust be determined fn each 
•£Wting and for each criterion that is being predicted. 

The construction of rating seniles has a long history which will not be 
lummarize^ here. As Thorndike and Hagen (1961) note, there are two bas.ic 

problems in obtaining sound ratings: (1) The willingness of the rater to rate 

■ ' V"' 

honestly and conscientiously in accordance with the instructions given to him; 
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and (2) his ability to rate consistently and correctly evfen with the best of • 
intentions. The implications of these problems are,' first of all, raters must 

9 

^(B "sold" on the importance of the ratings or else judgments may be hurried 
and superficial; secondly, attempts must be made to pr^ent the rater .from . 
being'Sverly influenced by^a general bias with respect to the person being 
rated. Specific attempts to solve these problems were -undertaken in the present 
study. ■ (^orrespondance with graduates and raters included cover letters from 
the Division of Nursing as well as the author: These letters fully described' 
the nature of and negd ^or the study and its relevanpe to nursing education. 
Raters were cautioned in their instructions regarding rating afccording to 
general impressions. All items contained pf-ov-isions for making "Not Applicable" 

■ , • THE BESI6N OF THE STUDY . . . - - . 

I 

The variables involved in the study included four indices' of GPA (Nursing 
GPA, Science GPA, Overall Delta GPA, and High School GPA), scores from the $tate 
Board Examinations in five areas (Medical, Surgjcal^ Obstetrical, Pediatric, 
Psychiatric), age, experience, seven subtest scores from the job performance 
rating scales. . . * , ; 

The question posed concerning potential strengths and weaknesses of the 
nursing program in this case involved an essentially descriptive analysis of the 
job performance ratings. It must be emphasized, however, that due to lack of 
a control group - i.e. ratings of a group of nurses with similar training and 
experience from other iristftutions - interpretation of these results must be 
cautious and tentative. It is quite possible. that some strengths or weaknesses 



as rated reflect perceptions of the rater which have little to dp with any 
partictflar training program, but may be based upon perceptions, of certain "types" 
of nurses., or nurses ofH certain age, or with other possibly non-relevant 
characteristics which the Delta nursing sample wliare in common with the general 
population of recent nursing graduates. In the pfesent study, the only control . 
available was utilized - i.e. ratings were obtained from several' sources who 
^ presumeably dp not share the same set of biases. Ratings were obtained from 
the graduate, from a doctor, and from a nurse who exercised some form of super- 
vision over the graduate.' 

The second question concerning the relationship among indices of GPA and 
Nursing Board Scores and rated job performance was investigated by a correla- 
•tional analysis. If moderate correlations of GPA and Board Scores were observed, 
the results would^e analyzed by step-wise regression procedures to determine. 
. the best combination of predictors to the criterion variable of job performance. 



•ERiC 



SELECTION OF SAMPLE * ' ■ • 

There were two considerations in the selection of the nursing graduate 
sample. First 'of all, the graduate should have been on the job sufficiently 
long to allow valid assessment of performance, and yet also be a recent 
graduate in- order to avoid contamination of ratings by lengthy experience and 
post-graduate training. Second, the size of the sample should be large enough 
to allow valid inferences. The decision was made toinclude all graduates of 
1970, 1971", and 1972. This yielded a population of 364 graduates. Initial . 
letters to 75 graduates were returned as uhforwardable, address unknown. 
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Initial contact was made with approximately 180 graduates. Of these, 111 / 
completed a rating scale while 93 were rated by a nurse. in a supervisory 
capacity and 57 were rated by a doctor. Atl graduates were included in the 
final analysis if one of the three rating§»was returned. This yeilded a 
totaVof 153 graduates although one or two of the ratings called for may-have * 
been itiissing. ^ . . 

Graduates who responded were found to work in a variety of nursing settings 
but prijparily in a hospital setting. It should be noted that graduates who 
responded were found to' have slightly higher but statistically significant GPA 
and Board Scores than non-respondents. 



, m RATING SCALES 

The 62 item rating stale completed by nurses in^a supervisory capaci'ty 
and doctors were identical. The rati>ig scale completed by graduate^ was also 
Identical except that the last 4 items and 1 item in the Leadership scale were 
deleted as not appropriate. Also the graduates were asked to rate the adequacy 
of their preparation at Delta rather than rating their perceived performance. 

Doctors and nurses who rated graduates wer^ nominated by the graduates 
themselves as the nurse or doctcfr who worked in closest supervision with them. 
These nominations were secured by the initiaV questiorw-ire sent to the 364 
graduates. Those nominated were contacted by mail, informed of the nature of 
the study, and requested to complete an open-ended questionaire. This question- 
aire asked them to^ define criteria of nursing performance which were most ^^ 
1mpoi»Jant-as they perceived their situation, , These collected criteria together 



with a statement of performance objectives provided by the Division of Nursing 
at Delta, served as the content base fir the rating scale items. The initial 
set of scale -Items written by the author^as revised and edited in accordance- 
with a review by faculty from the Division of Nursing which included a triaV 
administration of all scales by a number of individuals not in the designated-, 
graduate population. - , " . 



ANALYSIS OF THE DATA i. 

The statistical analyses was performed with tlie assistance of the Statis- 

tical Package for the Social Sciences (SPSS), a system of computer programs 

' — ■ < 

available at the Computer Center at the University of Northern Iowa, ^^^JffTtlfe^ 
present study, a^complete set of data for each pf^e 153- graduates in the 
sample Wds-Hwt available. In order to njake max'imum use of available data, a 
case was deleted from the analysis only when the missing data was required . 
for tfhat specific analysis. In the case of correlations, missing data from 
either pair caused pair-wise deletion for that specific correlation* Analysis 
of all rating scales was "conducted in a similar fashion. The mean response 
for eacih item was computed and represents, the^ weighted. sum ofA responses to • 
the item. The categories were weighted from I to 5 with 1 being poor and '5 
being excellent. The N/A category as well as items not responded to were in 
all cases ignored by the analysis. .Subtest means were competed by first 
summing within cases the weighted item responses in the subtest and dividing 
by the number of Items in the subtest. These were summed across ^ses and 
divided hy the number of cases to get the subtest'mean. 

♦ 

' • . • . 12 ;. 



\ . ' ' " ^' RESULTS m DISCUSSION ' ' - ' . 

•The ^reporting atid '^discussion of results hasEjbeert''si|>ar"^ited in accordance 
with the- two basic questlqn^ .pbsed by/ the siu'dy;--.R^tiH^ dtscusston. orf: • 
possibl€ strfengths and. weaknesses are taken up first and followed by results 
and discussion of the relationship of GPA and Board -Scores with rated job 
perfortn^n.ce. A final' section discusses the merits of the rating scale with 
possible suggestio'ns or reviston for future use, ' t. 



POTOf lAL STRENGfHS AND WEAKNESSES 



It -should be noted again that dije to^lack of a control. group of graduates 

from other jiursing^ programs, results should in no way be .i nterpreted as pla<;^ing 

^ ' \" * \ ** * 

graduates or* thep5ursi!lg program pn a good-bad or strong-weak contjinuum relative 

to, other graduates or other programs. It is quite possible that an area 

designated "weak" .in the pre^sent context could Ije r*epresentative of ajl recently 

'.graduated nurses and that if^data collected on a representative . sampi e . of ,al 1 

'recent graciuates m,ight demonstrate *thSt an area designated "veak" in the. present 

•study is a "strong" area relative to otheh institutions. . Of course, tne opposite 

could also be, true. The fact of the matter is that* the present data" cannot be 

usfed to.substantiate any such external comparisons. Assessment of ^strength^ and 

weaknesses in the preseht .study refer^only to relative rated perforiiance^ of 

Delta graduatesv * , , ^ / 



RESULTS AND DISCUSSION , 

Averaging of subtest rank according to the three ratings indicate basic 
agreement among graduates, nurse supervisors, and doctors that^ areas of strength. 
Include those of "implementing nursing care" and "interpersonal relations" while 
areas of weakness include "leadership and ynit procedures" and "profess^nal 
involvement." While thq[ category, of professional involvement was rated relatively 
•high by doctors, fifty percent of the doctors checked the Not/Applicable category. 

Inspection of individual stems in thB. scale indicate agreement of strength • 
in the following areas: identificatioli of -patKnt needs, providing nursing care . 
until medical orders are/obt*4ned,-pr^^r1ng pattenta for diagnostic procedures, 
and kindness arid consideration shown toward patients. Agreefftent, of weak. areas 
fftclude use of community resources, application of facts and principtes-/of ^ , , ^ 
nutritiortjj'^Koviding f6r a group of patients, conducting nursing care conferences^; 
assuming leadersh-ip and management responsibilities, and assisting in the 
instruction of auxiliary personnel. It should be noted tf»at with the exception- 
of the item, concerning leadership and management these "weak" items were checked 
as Not/Applicable by a relative] y^W,gh percentage of raters. . T 

Certain items show substantial disagreement amtitig ralers »Kkh may reflect 

differences of professional perceptions. Graduates felt relatively well" prepared 

■ , ■ ■ " - - ' . . ' '•' y , 

to develop a written plan of car^ for the patient, however, their supervisors^ / ■ 

feel they don't perform that task well. Nurse supervisors were relatively well/ 

satisfied with gr adu a te)s- application of lanatomy- and physiology, while doctor^ 

and graduates were qu.ite dissatisfied. Two other items are. of particular interest 

\ . ' ' •• • . 

ip that graduates rated their preparation very low \j\ the areas of taking . 



appropriate actions in'emergencies and .<:arry^ng out commoply occurring techniques 

; ■ ■ • :'.„ . . / •. />- / , > ■ i . -' 

for patient care. Quite the contrary' say Supervising' nursje's and .doctors who gave 

these 'Hems very high ratings.- Finally,' consultation :anfl:|cooperati on with . 

physicians and otherS ranked, very high"w1/th docto^ybut;<r^5py low'wlth nurse 

supervisors. ^ Graduates perceive th;rs as an area needj'l^further deyelopmeot In 

their preparation. / >. / /■''.\.(/, -'' Js^ 
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. ' , . . • . \L ' *■ / 

RESULTS/AND -DISCUSSION OF OPEN^ENtiiEO- QUESmONS^ . ^ 

» ' ' ^ / ** ^ / / ,-~ »\ 

GRADUATE RESPONSES ' ' ' ^ ^' ' ' : . '^>5V:> -.^y- '' 

Fifty-five'p6r cent of the' graduates .mentix)^jeg^^.^^ -j ; 

" ' . J '"'^ -vX '\. ■' '/ ^' - •'- -• ■ ,'- 

, (ATL) as a nwjor strehgth of tKe^irogram.v^^twenty^five per cent of the graduates 

felt that -tKe exceTl'^t^e^of -some ^of .thfeir instructors was a major strength. ■"'^ 

y- s. . Other /strengths which weriBfm6ati6ti€*d"by^t^0east ten per cent of the graduates 

z;^^ : ^z-- were: ' 'go^d iristrucJLU)n Irv prtncipiiSsxand tneoRies df nursing, weekly quizes, 

.;;\th^ stfres^ Oil Nursing Car^ Pilaris^,' and ijthe tJ$e :oY-^»^^ " 

• / , - •, "NeSirly aTl graduatesi Irid^cVtfed tl\at:th'ey f^lt tha|vmore clin1<:aV^xperience 

wasXneBded,; espepl.iklly. ex'peri^nc^ allow tota\mvolvemerrtxx Lfkt^lise 

sX'"-"/ '. Viearly ,al r:graa,uates felt the ^neferf !f or additional cours^^Wbrk, Specially in 

. " '4 At.&nk bf Dharmacoloav. Dhvsioloav. anrftrtmv. and niJt.irit.ii)n. ' nnhnict.iinStv 



X ••\v?5i)m€';suc||e!sJi1ons.w^^ also maiie concerning changes in 'the il earning 6fi\^1.ronment. 

* -X'ij,{j^?j"cl'»itf£d i^^^^^ selecl>;lM'!^f?'i Instructors,, smalliw:, .s^udent-instruttor ratio,. 

'•■•"Xt'*.'/^i-y Xr ■'.■^j^r. ■ 'X^'MXi.X- . X .X...;:'X • ■' ■ . 

,r^j^^^' • ^I^^^H't method ^6?^eS'f>Tvifig. studentri.^istrllctpr conflic^. Finally, 




per. cent of . the gra^duates iwhp :f;e1t/t& the otieh^dopr. admissions po.lier should 
be ended and the course of studii^s -extended, io a ^rV^^ • 



/,• ■ 



NURSE' SUPERVISOR AND .DOCTOR RESPONS 

■ InterpersonaT,,^e1 anions were,seen by" nur^e supervisors equally as a strength* 
orweakness= of the graduate. tjjS^s ■informatioh.aifc' ■Pirst glance would not appear ' 
to be parti cular^^l nforoa^ti^e. CVose iftspec^tjori of the cornments, however, suggests 
tha't interRefs()Hat>elations des^ignated as a -strength referred to relations with 

'^patients, farniT.ies^; or hospital piersonnel aij the occupational level of the 
graduate -or JbeTbw? -/Interpersonal relations mentioned as a weakness typically 

. r6>erred to^the relationship between the'*graduates and their superiors. On thq 
otKen handrdoctors see the relationships of graduates with patients and sub- 

^flllrrat^s as a major strength with no mention of weakness- in dealing with ' 
superiors. These comments substantiate the contrasting viewpoints of nurse 

i supervisors and doctors in one of the itetns in the rating -scale. > v , ^ v 

Nurse supervisors frequently mentidnecl continued efforts at improvement of 
slef and nursing, knowledge as a strength but bith nurse supervisors and doctor,s ' 
perceive a -need for contiTiued education. to improve technical skills and keep 

i * * * ' ' ' ^ ^ ' 

knomedge current; ^ 

Leadership was frequently mentioned by^rse supervisors as a. strength for 

sortte graduates but a weakness of others. Approximately fifty per cent of the • 

nurse supervisors mentioned 'the. leadership area as a strength or weakness which 

»-•■*■ ' . . ' ■"."■.* 

seems to Indicate leadership as an important performance variable for many of 

the graduates occupational categories. * ^ "'^ • ' 



ft 

t 

• * ♦ 

Self confidence J organizational efficiency, decision jnaking ability, and 
ability to set priorities were seen as weak areas by nurse supervisors and 
doctors. It would seem that these performance variables relate most directly to 
clinical experience. - * , 



SUMMARY 

' \ ' ■ ■ 

Taken as a. composite, responses to the rating scale and to questions 
concerning major strengths and weaknesses may indicate the follt)w1ng generaliza- 
tions concerning the present samp.\.e of Delta nursing graduates of T970, 1971 , 
an{n972: ^ • 

1. The ATL lab^^as perceived by graduates' to .be an effective^ • 
. - ■ Instructional ^ald. ^ 



2. Graduates perceived themselves as providlhg comp&ssiJonate and. 
' und^rsjAndihg care to pftf;ients and able to rebate w4ll with 

patients And able to relate welTwith patients and subordinates. 
This perception was generally substantiated by tQeif raters* 

3. Graduates may'have difficulty in relating well with? superiors, 

. especially nurse sup(§r1ors although information 'from doctors 

' ■/ ■ ' . ' 

: ' does not substantiate this generalization. (Given the fact . 

' ^ that doctor raters were nominated 'by the graduates and/return. 

rate from doctors was low it is possible that the ratings • 

obtained from doctors may not be representative.) 

ERIC _/ 17 . • v 



Graduates perceiverf a need for additional clinical experience which 
allov^^for the total involvement of the graduate. This lack of 
clinioFexperience may' be substantiated by ratings from nur^ 
supervisors and doctors who perceived the lack of clinijE^ experience 
in terms of lack of self-coiifidence, lack of orgajmation and setting 
' of priorities, andvlack of leadership capabt^ies. 

■ 

Graduates are often placed in positfiJ^s of leadership. Some 
graduates were rated well in tKis area, others poorly. Graduates 
perceived a need for leadership experience and possibly leadership 
classes in their nur^thg. preparation, . ' ^ 

Graduate* perreived themselves as lacking some basic technical 
skills bfit/th^s did not-^appear to be entirely substantiated by 



thei r superv i sor s . - 



V 



Graduates perceived the need fpr additional coursework especially^ 
m the areas pharmacoJogyV physiology, anatomy, ani nutrition, ; 
Boctors tended to agree with this assessment but it was less 
evidept in the returns from niirse supervisors, ^t is prob^lbe 
that the. particular -course perceived as "needed" is one which 
rebates to the parti cdlar -job situation in which the ^ra.duate works. 
^There is d'greemejat, however, that continuing education is needed 
for graduates/ 



GW\ AND BOARD SCORES AS PREDICTORS OF 'PERFORMANCE ^ 



RESULTS 



Tables of correlations between GPA and age with the three ratings are 
presented in Table 1.1 Graduate ratings of the program show low positive cor- 
relations with nurs^g GPA and age. A similar relationship between age and 
nuVsing GPA was notedlin an earlier study by the Office of Research and Develop- 
meriit at Delta College.! The correlation of .24 found between nursing GPA and 
gra(Iuate ratings .suggests a slight tendency for graduates" with high GPA to- rate 
thelprogram more favorjibly. ■ . . " 



Table 2 contains correlations between the three ratings, experience and 



the five areas of State 



Nursing Board scores. Experience was operationally 



defiried as mdnths of work experience as a nurse. Experience. ai a licensed 

' ( • ; • • , 

practical nurse was included.. Negligible cprrelations among these yariables 
ranged fl-om -.15 to +\.14.* , . v~» 

Correlation^ among the three rati*hgs were quite I'pW and not statistically 
significant. A nega'tijVe correlation between dytor''^* rating and experience is 
worthy of note (r = r.i28) and ^s discussed below. v 



DISCUSSION 

The data suggest no significant linear relationships between measuresN^ 
scholastic success and the job perfonpance of nursing graduates as rated by 
nurses and doctors who work with- the graduates. As noted in the discussion 
of strengths and weaknesses, there'are Indications that doctors" who work "with 
graduate's perceive job performance from a different perspective than do the^ 

"nurses who 'rated the graduates. This may account for some portion of the low 

- . ^ . J. 

correlation between doctor and nurse ratings (r = .14, |<>= 31). If the 
predictor variables of GPA and Board scores had demonstrated a positive 
correlation with doctor ratings as well as nurse supervisor ratings,- some 
sigrrificant relationship might have been produced by pooling the ratings of 
doctors and nurse supervisors into a composite rating. Such a procedure has 
been shown to enhance f^e 'reliability of ratings (Thorndike and Hagen, 1961), 
but does not appear to be warranted in the present case. 

The significant negative torrelation (-.28) found between doctor's 
ratings and length of experiences is 'somewhat surprising. The factors which 
contribut^ to this negative relationship are a matter of conjecture but may 
reefnphasize the point that. personal attributes and Jiersonality dimensions 
influence perceptions of job, performance. 

Given the fact that graduates work in many diffjerent settings with different 

demands placed upon their capabilities., it is possible that scholastic success 

■ *'.»• 

may show a relationship with some types of nursing oQcupations, but not others. 
As Goldman (1961) observes, not all members of a particular group are equally 
predictable and by sub-dividing a group in certain wajjs we can sometimes raise 
the efficiency of p'redictfon for at^-Teiast some of its members. 



' TABLE 1 

INTER-CORRELATIONS OF. AGE AND" G PA WITH RATINGS 



Nursing 
GPA 



Science 
GPA 



Overal 1' 
GPA • 



Age 



High School 

IGPA 



Doctors ' 
Ratings 



Nurse 

Supervisor 

Ratings 



.07 



.04' 



.14 



.01 



..-.03 

:02 



-.04 



.06 



.02 



Graduate 
Ratings 



.24* 



-.12 



.10 



.20** 



f.-.02 



**sig ;;oi 



TABLE 2 ■ 

INTER-CORRELATIONS OF STATE BOARD SCORES. 
WITH EXPERIENCE AND RATINGS - 



Nurse 

Supervisor 
Rating 



Graduate . 
Rating 

Experience 



ERLC 



Medical Surgical Obstetrical. Pediatric Psychiatrijc 



-.10 

-.09 
.06 



.01 " 
-.08 

;oo 



-.07. 

•-.Ol" 
.,04 



sig .05 




-.01, 

•15 
.14* 



Inspection of. rating scalies as welT as subtest intercorrelatlons. suggest' 
that a , halo effect governed sowe^of the ratings esptecially those completed by 

the doctors. The halit^ effect is the tendency to- rate in terms over-all . 

\ _ ■ ' ■•■ . . ' ' 

general impression wijkh6ut~differentiatlng specific «s,pects.= In the present 

case, this was suggestiid wheA some ratings we're returned with^ virtually all 

'A ^- - . - " . , • 

items checked in- the same category. . ; . 



SUMMARY • . " .. ' / , • . 

\, ' , ^ i a - ■ ■ ' . . •, 

Results of the present Jstudy suggest little or no releitionship between GPA, 

Board scores, and rated job performance. These results are pot inconslstant 

with a number of studies cited, above concerning graded and«occuf)atioha1 performance. 

There are^ number of possible ^explanations for such results-. Eor instance, one 



J. 



might question the reliability of 'not only the rating scalg,, ,but also the indices 

of GPA and the Board scores as wetl. An impg^rtant considjeration'in the evaluatldn 

of the rating scale is that' of inter-rater reliability.; Aniinbiased estimate of 

nter-rater reliability requires two eqqally qualified raters w^ho. bear essentially 

the same relationship to the ratee. ' In. the present case, th'ls was not. possi brie - 

• • •-• ' . ■ . - 
and no unbiased estimate can be made of> the inter-rater .relialiility of the scale. 

• . ,• • ^ 

Inspection of the responses to the ofefi-ended questions regarding strengths 
and weaknesses suggests that a number of fafctors weigh lieaviiy In assessment of 
job performance which probably have little tp, dojwith ability to^get grades or . 
perform well on Board scores. These includef|variaBles such as interpersonal, ^ ,. 
relations, dependability, enthusiasm, reactlbns under stress, .prof essiona,l appear* 
ance, punctuality, adaptability, and decisiveness^^ , ' . . 



^ > 



The following generalizations appear to be warranted by the results: 

1. There was no demonstrated relationship between measures of 
scholastic success such as fePA and State Board scores with - 

' .-rated job performance 0/ Delta graduates, 

2. The inter-rater reliability of the scale may iieed improvement 
for future use although an unbiased estimate of inter-rater 
reliability was not obtained in the studyj 

3. Rated job performance appears to be greatly influenced by 

a number of personality traits. Prediction of job performance 
* might be enhanced by attempting to estimate th6se attributes 
during training. 

4. Ratings, especially those ratings completed by doctors, appear 
, to have been influenced by a halo effect. « 



23 



RECOMMENDATIONS 

The following recommendations were provided to the Nursing division at 
Delta College with acknowledgment that (1) appropriate curricular changes may 
already have been undertaken (2) suggestions for implementation are not (and 
probably can not) be specific. ^ 

1. ) It was recommended that the Division of Nursing investigate the 

feasibility of incorporating additional clinical e)9perience into 
the" nursing program. 

# 

2. ) That ^he Division of Nursing investigate the feasibility of 

incorporating leadership training and/or experience into.the 
nursing program. 

3. y That advanced coursjas in the area of pharmacology, physiology, 

anatomy, and nutrition may be needed as optional courses or 
post-graduate courses. 

4. ) That the present data to be used. as a baseline for evaluation - 

of the^effectiveness of curricular changes. 

5. ) That the rating scale be revised for future use. 

6. ) That further efforts ^to predict job performance include 

personality variables- in the prediction. - 



7.) That recornnendations of graduates for jobs and post-graduate 
study be based on information in^addition to GPA and Nursing 
Board Scores,^ \ 

8J .That attempts be made to make the scale available to otfier nursing 
ainsti tut ions that comparative data may bfe gathered. 
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